Functional Testing, Structural Testing, and Code Reading: What Fault Type Do They Each Detect?

The origin of the study described here is the experiment performed by Basili and Selby, further replicated by Kamsties and Lott, and once again by Wood et al. These experiments investigated the effectiveness and efficiency of different code evaluation techniques (functional and structural testing and code reading). The working hypotheses are the same in all three experiments, although some experimental conditions were changed. The experiments described here use the experiment package elaborated by Kamsties and Lott and examine some of the questions posed as a result of these experiments. Wood et al. concluded in their replication of the original study that the relative effectiveness of the techniques depends on the program and fault type. In fact, they suggest formulating a fault taxonomy based on technique sensitivity. Our study intends to compare the relative effectiveness of the testing techniques and to relate the testing techniques to fault types.